Justice V Gopala Gowda recused himself from the hearing without citing any reason and said, "let the matter be placed before the Chief Justice" for its allocation to some other bench.
The Supreme Court on Monday kept its options open on examining the dropping of conspiracy charge against accused like L K Advani, M M Joshi and Uma Bharti in the demolition of the disputed Ram Janmabhoomi-Babri structure in Ayodhya in 1992.
The SC asked special judge to control the proceedings in accordance with law so that inordinate delay that is beyond the time-frame is no longer breached.
The CBI court also ordered framing of charges against the accused in light of the Supreme Court order dated April 19, 2017.
The top court, which had earlier fixed August 31 as the deadline for completion of proceedings including pronouncement of the verdict, took note of the report filed by special judge S K Yadav in the high-profile case.
The court also observed that late Vishwa Hindu Parishad leader Ashok Singhal wanted to save the structure because Ram idols were inside.
The Vishwa Hindu Parishad has too decided against observing the 27th anniversary of the demolition as "Shaurya Diwas".
The apex court will also decide whether the trial of the VVIP accused can be transferred from a court in Rae Bareli to Lucknow.
The twin towns of Ayodhya and Faizabad were on the edge on Saturday as tension mounted on the eve of VHP's yatra with the UP government cracking down on the saffron outfit arresting some of its leaders and over 350 activists.
Uneasy calm prevailed in Ayodhya, which on Saturday looked like a fortress, with deployment of more than 10,000 cops on the 22nd anniversary of the demolition of the Babri Mosque.
There are indications that the Modi dispensation was disinclined to bring the VHP into the temple-mosque frame.
'He couldn't hold himself from chanting 'Siyavar Ramchandra ki Jai'.'
'Mohan Bhagwat is very upfront with his views and arguments.' 'Mohan Bhagwat is competent to deal with Modi's personality cult.'
The event brings to fruition the Bharatiya Janata Party's 'mandir' movement that defined its politics for three decades and took it to the heights of power.
Iqbal Ansari, one of the main plaintiffs who fought for the right of the community over the disputed land, says Muslims in Ayodhya won't be satisfied if the Sunni Waqf Board accepts the five acres in Dhannipur.
'Gods of different religions haven't warred, so we shouldn't either.'
The apex court, however, noted that Rajasthan Governor Kalyan Singh enjoys Constitutional immunity and can be tried only after he ceases to hold the office.